1. Field of the Invention
This invention pertains to computer systems and more particularly is concerned with artificial neural networks.
2. Related Art
Artificial neural networks (or in short neural networks) are powerful information processors. Neural networks are increasingly being applied today to solve problems in such diverse areas as speech recognition, expert system, process control, and robotics. They have a revolutionary potential exceeding far beyond their current capability. In 1969, Minsky and Papert proved that 2-layer perceptron networks were inadequate for many real-world problems such as the exclusive-OR(XOR) function and the parity problem which are basically linearly inseparable functions[MiPa69]. Kolmogorov and Stone-Weierstrass theorems have proved that three-layer neural networks can perform all real mapping. However, there has not been yet found a learning algorithm which can synthesize a three-layer threshold network (THTN) for an arbitrary switching function.
Recently, the Back-Propagation Learning (BPL) algorithm has been applied to many binary-to-binary mapping problems. Since the BPL algorithm requires that the activation function of a processing element (i.e. neuron) be differentiable and the activation function of a threshold element is not differentiable, BPL a algorithm can't be used to synthesize THTN a for an arbitrary switching function. Moreover, since the BPL algorithm searches the solution in continuous space, the BPL algorithm applied to binary-to-binary mapping problems results in long training times and inefficient performance.